Singular Value Perturbation and Deep Network Optimization

Relative Perturbation Theory: I. Eigenvalue and Singular Value Variations∗

The classical perturbation theory for Hermitian matrix eigenvalue and singular value problems provides bounds on the absolute differences between approximate eigenvalues (singular values) and the true eigenvalues (singular values) of a matrix. These bounds may be bad news for small eigenvalues (singular values), which thereby suffer worse relative uncertainty than large ones. However, there are...

Updating Singular Value Decomposition for Rank One Matrix Perturbation

An efficient Singular Value Decomposition (SVD) algorithm is an important tool for distributed and streaming computation in big data problems. It is observed that update of singular vectors of a rank-1 perturbed matrix is similar to a Cauchy matrix-vector product. With this observation, in this paper, we present an efficient method for updating Singular Value Decomposition of rank1 perturbed ma...

An Optimization of Deep Neural Networks in ASR using Singular Value Decomposition

Restructuring of deep neural network acoustic models with singular value decomposition

Recently proposed deep neural network (DNN) obtains significant accuracy improvements in many large vocabulary continuous speech recognition (LVCSR) tasks. However, DNN requires much more parameters than traditional systems, which brings huge cost during online evaluation, and also limits the application of DNN in a lot of scenarios. In this paper we present our new effort on DNN aiming at redu...

Singular perturbation theory

When we apply the steady-state approximation (SSA) in chemical kinetics, we typically argue that some of the intermediates are highly reactive, so that they are removed as fast as they are made. We then set the corresponding rates of change to zero. What we are saying is not that these rates are identically zero, of course, but that they are much smaller than the other rates of reaction. The st...